Papers with softmax transformation
Sparse Sequence-to-Sequence Models (P19-1)
Copied to clipboard
| Challenge: | Sequence-to-sequence models are dense and assigning nonzero probability to implausible outputs. |
| Approach: | They propose a new family of -entmax transformations that includes softmax and sparsemax as particular cases and is sparser for any > 1 . they provide fast algorithms to evaluate these transformations and their gradients, which scale well for large vocabulary sizes. |
| Outcome: | The proposed models are able to produce sparse alignments and assign nonzero probability to short list of plausible outputs, sometimes rendering beam search exact. |